Information processing apparatus, remote indication system, and computer readable medium

ABSTRACT

An information processing apparatus includes: a reception unit that receives multiple captured images that are captured with a capture device in parallel from an external apparatus to which the capture device that captures an object is connected; a detection unit that detects a priority image that is included in the multiple captured images being received by the reception unit and is assigned priority information that specifies the priority image to be preferentially displayed on a display device; and a controller that controls the reception unit to stop receiving other captured images which are not detected by the detection unit and to receive the priority image detected by the detection unit and controls the display device to display the priority image on the display device when the priority image is detected by the detection unit.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based on and claims priority under 35 USC 119 fromJapanese Patent Application No. 2008-261106 filed Oct. 7, 2008.

BACKGROUND

1. Technical Field

This invention relates to an information processing apparatus, a remoteindication system and a computer readable medium.

2. Related Art

There has been conventionally known a remote indication system includinga server (e.g. a computer) with which a video camera and a projector areconnected, and a client (e.g. a computer) of a remote place connectedwith the server via a network.

SUMMARY

According to an aspect of the present invention, there is provided aninformation processing apparatus including: a reception unit thatreceives multiple captured images that are captured with a capturedevice in parallel from an external apparatus to which the capturedevice that captures an object is connected; a detection unit thatdetects a priority image that is included in the multiple capturedimages being received by the reception unit and is assigned priorityinformation that specifies the priority image to be preferentiallydisplayed on a display device; and a controller that controls thereception unit to stop receiving other captured images which are notdetected by the detection unit and to receive the priority imagedetected by the detection unit and controls the display device todisplay the priority image on the display device when the priority imageis detected by the detection unit.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the present invention will be described indetail based on the following figures, wherein:

FIG. 1 is a block diagram showing the structure of a remote indicationsystem in accordance with an exemplary embodiment of the presentinvention;

FIG. 2 is a block diagram showing the hardware structure of PC 6;

FIG. 3 illustrates the data structure of image data received by PC 6;

FIG. 4A illustrates an existing reception condition of image data, andFIG. 4B illustrates a reception condition of image data of thisembodiments;

FIG. 5 is a flowchart showing a process executed with PC 6;

FIG. 6 is a flowchart showing an image reception process of the step S3in FIG. 5;

FIG. 7 is a flowchart showing a request process of a second image data;

FIG. 8 is a flowchart showing an interruption process executed during adisplay process of the step 4 in FIG. 5;

FIG. 9 illustrates an example of a message 91 displayed in first window13;

FIG. 10 is a flowchart showing a process executed with a video server 2;

FIG. 11 is a flowchart showing a process executed with a distributionserver 3; and

FIG. 12 is a flowchart showing a process executed with a drawing server4.

DETAILED DESCRIPTION

A description will now be given, with reference to the accompanyingdrawings, of exemplary embodiments of the present invention.

FIG. 1 is a block diagram showing the structure of a remote indicationsystem in accordance with an exemplary embodiment of the presentinvention.

The remote indication system in FIG. 1 is provided with a server system1 corresponding to a first information processing apparatus, and a PC 6functioning as a second information processing apparatus. These elementsare connected to each other via a network 5 functioning as means ofcommunication. The server system 1 is provided with a video server 2, adistribution server 3 connected to a first camera 7 and a second camera9, and a drawing server 4 that is connected to a projector 8. Based on acontrol command from the drawing server 4, the projector 8 emits lightbeams or projects an annotation image onto an object 11 (e.g. a screen)via a half mirror 10. It should be noted that the annotation imageincludes an image of any types such as a line, a character, a symbol, afigure, a color, and a font.

The first camera 7 is composed of a video camera, captures a reflectedimage of the object 11 via the half mirror 10, and outputs the capturedimage to the video server 2. More specifically, the first camera 7captures the overview image of the object 11. It is better that an angleof view and an optical axis of the first camera 7 are identical withthose of the projector 8, because the half mirror 10 can work with thefirst camera 7 ant the projector 8.

The second camera 9 is composed of a network camera or a camera with apan tilt zoom function, and is moved freely by an operator. The secondcamera 2 outputs the captured image to the video server 2. And thesecond camera 9 is used for enlarging and capturing a part of the object11. More specifically, the second camera 9 captures a part of the object11.

The video server 2 stores the captured images by the first camera 7 andthe second camera 9. The distribution server 3 distributes the capturedimages by the first camera 7 and/or the second camera 9 to the PC 6based on a distribution request of the captured image from the PC 6. Thedrawing server 4 acquires the annotation image from the PC 6, andoutputs it to the projector 8. It should be noted that the server system1 may be composed of one server with functions of the video server 2,the distribution server 3, and the drawing server 4 although the serversystem 1 is composed of several servers in this embodiment.

A display device 12 is connected with the PC 2. The display device 12displays a first window 13 displaying the captured image by the firstcamera 7, a user interface (UI) 14 to draw the annotation image, and asecond window 15 displaying the captured image by the second camera 9.The UI 14 includes a group of buttons such as a pen button, a textbutton, and an erase button, and icons defined by lines and colors. Forexample, when the pen button of the UI 14 is pushed down and a figure isdrawn on the object 11 in the first window 13, the figure data is outputto the projector 8 from the PC 6 via the drawing server 4. The projector8 draws the figure on the object 11 based on the figure data.

In FIG. 1, the second window 15 displays a display area 17 displaying anobject 16 which is a part of the object 11, an operation button 18, azoom in button 19 and a zoom out button 20. It is unnecessary that thesecond window 15 is displayed all the time. The second window 15 isdisplayed on the display device 12 when a predetermined command is inputto the PC 6. When the operation button 18 is pushed down, the secondcamera 9 changes its direction based on the direction that the buttonpushed down indicates. At this time, the image displayed in the displayarea 17 is changed depending on the movement of the second camera 9.When the zoom in button 19 or the zoom out button 20 is pushed down, theimage displayed in the display area 17 is enlarged or reduced.

And the PC 6 outputs a control command to the projector 8 via thedrawing server 4, and controls a brightness of the projected image bythe projector 8. Also the PC 6 outputs the control command to the firstcamera 7 and the second camera 9 via the video server 2 and thedistribution server 3, and controls the capturing angle and thebrightness of the captured image by the first camera 7 and the secondcamera 9.

It should be noted that the remote indication system can be providedwith several clients (PCs) although only the PC 6 is illustrated as aclient in FIG. 1.

FIG. 2 is a block diagram showing the hardware structure of the PC 6.

Since the hardware structure of the PC 6 is the same as the eachhardware structure of the video server 2, the distribution server 3, andthe drawing server 4, a description will now be given of the hardwarestructure of the PC 6 herein after. It should be noted that, in FIG. 2,the reference numerals 21 to 29 designate the elements of the videoserver 2 and the reference numerals 31 to 39 designate the elements ofthe distribution server. And the reference numerals 41 to 49 designatethe elements of the drawing server 4 and the reference numerals 61 to 69designate the elements of the PC 6.

The PC 6 includes: a CPU 61 (a reception unit, a detection unit, acontroller, a second reception unit) that controls the entire PC 6; aROM 62 that stores control programs; a RAM 63 that functions a workingarea; a hard disk drive (HDD) 64 that stores various information andprograms; a mouse and keyboard 65; a network interface 66 that isconnected to other computers; a video interface 67 that is connected tothe display device; and a USB (Universal Serial Bus) interface 68 thatis connected to a USB device, not shown. The CPU 61 is connected to theROM 62, the RAM 63, the HDD 64, the mouse and keyboard 65, the networkinterface 66, the video interface 67 and the USB interface 68 via asystem bus 69. It should be noted that a CPU of a server combining thevideo server 2, the distribution server 3 and the drawing server 4corresponds to a first reception unit and an addition unit.

FIG. 3 illustrates the data structure of the image data received by thePC 6.

As illustrated in FIG. 3, the image data received by the PC 6 arecomposed of a data header 51 with header information, and a data body52. The data header 51 contains attribute information such as a capturetime, a type of the image data and a size as needed, and may contains apriority tag (priority information) as described hereinafter. The databody 52 is real data of the image.

FIG. 4A illustrates the reception condition that a known apparatusreceives the image data, and FIG. 4B illustrates the reception conditionthat the remote indication system in accordance to this embodimentreceives the image data.

As illustrated in FIG. 4A, when the image data are distributed in orderof a first image, a second image and a third image from the distributionserver 3, the PC 6 receives the image data based on that order. Inaccordance with this embodiment as illustrated in FIG. 4B, the PC 6requests to receive the data header of new image data whether or not thePC 6 is receiving the image data. And when the data header contains thepriority tag, the PC 6 cancels receiving the image data the PC 6 isreceiving, receives the image data of the image (priority image) thathas the data header containing the priority tag (priority information),and displays the priority image on the display device 12. When there isnot the image data the PC 6 is receiving, the PC 6 receives the imagedata of the priority image, and displays it on the display device 12. Inthe example illustrated in FIG. 4B, the PC 6 receives the second imagewith the priority tag, and displays the second image on the displaydevice 12.

FIG. 5 is a flowchart showing a process executed with the PC 6.

First, the CPU 61 of the PC 6 establishes a connection with thedistribution server 3 and the drawing server 4 (step S1).

Then, the CPU 61 receives a command to distribute the image from thedistribution server 3 (step S2), and starts an image reception process(step S3). A detail of the image reception process is describedhereinafter.

The CPU 61 executes a display process to display the received image datain the first window 13 or the second window 15, and a drawing process toreflect the annotation image drawn with the UI 14 to the first window 13(step S4).

The CPU 61 determines whether there is a projection request thatrequests to project the annotation image drawn in the first window 13(step S5). When the determination of the step S5 is YES, the CPU 61sends the annotation image drawn in the first window 13 to the drawingserver 4 as a command (step S6), and moves to the step S7 describedbelow.

When the determination of the step S5 is NO, the CPU 61 determineswhether there is a termination request (e.g. a request to shut down apower) (step S7). When the determination of the step S7 is YES, thepresent process is terminated. Meanwhile, when the determination of thestep S7 is NO, the CPU 61 goes back to the step S2.

FIG. 6 is a flowchart showing the image reception process of the step S3in FIG. 5.

Here, the image data to be displayed in the first window 13 aredescribed as the first image data, the second image data, and the thirdimage data in order of the distribution. And 001, 002, 003 are added toeach of the first image data, the second image data and the third imagedata as an identifier.

First, the CPU 61 sets the identifier of the first image data (it means001) as the identifier of the image data received from the distributionserver 3 (step S11), and sends the request of the first image data tothe distribution server 3 (step S12). Then the CPU 61 starts to receivethe response that means the first image data from the distributionserver 3 (step S13).

The CPU 61 updates the identifier of the image data received from thedistribution server 3 from the identifier of the first image data to theidentifier of the second image data (step S14), and executes the requestprocess of the second image data (step S15).

FIG. 7 is a flowchart showing the request process of the second imagedata.

First, the CPU 61 determines whether it is receiving the terminationrequest (e.g. a request to shut down a power) (step S21). When thedetermination of the step S21 is YES, the present process is terminated.Meanwhile, when the determination of the step S21 is NO, the CPU 61sends the request of the second image data to the distribution server 3(step S22). It should be noted that the second image data is capturedafter the first image data, and is newer than the first image data.

The CPU 61 starts to receive the response that means the second imagedata from the distribution server 3 (step S23). Then, the CPU 61determines whether the priority tag is included in the data header 51 ofthe second image data (step S24). The priority tag is the tag to specifythe image data to be displayed next after the image data being displayedin the first window 13.

When the determination of the step S24 is NO, the CPU 61 updates theidentifier of the image data received from the distribution server 3from the identifier of the second image data to the identifier of thethird image data (step S25), and goes back to the step S21. In the loopprocessing from the step S21 to the step S23, the CPU 61 receives thedata header 51 of the new image data after the data header 51 of the newimage data, and determines whether its data header 51 includes thepriority tag.

When the determination of the step S24 is YES, the CPU 61 stopsreceiving the first image data, the process of the step S13 in FIG. 6(step S26). And the CPU 61 registers the second image data to the HDD 64as the image data to be displayed in the first window 13 next (stepS27), and moves to the step S16 in FIG. 6.

The CPU 61 determines whether it is receiving the request which requeststo cancel the image reception process in FIG. 6 from the mouse andkeyboard 65 (step S16).

When the determination of the step S16 is YES, the present process isterminated. Meanwhile, when the determination of the step S16 is NO, theCPU 61 determines whether the reception of the image data with thepriority tag has been finished (step S17). When the determination of thestep S17 is NO, the CPU 61 goes back to the step S16. Meanwhile, whenthe determination of the step S17 is YES, the CPU 61 stops the requestprocess of the second image data (step S18).

Lastly, the CPU 61 determines whether it is receiving the terminationrequest (e.g. a request to shut down a power) (step S19). When thedetermination of the step S19 is YES, the present process is terminated.Meanwhile, when the determination of the step S19 is NO, the CPU 61 goesback to the step S11.

FIG. 8 is a flowchart showing the interruption process executed duringthe display process of the step S4 in FIG. 5.

First, the CPU 61 of the PC 6 determines whether it has received theimage data with the priority tag (step S31). When the determination ofthe step S31 is NO, the CPU 61 repeats this determination. Meanwhile,when the determination of the step S31 is YES, the CPU 61 displays themessage 91 showing that the CPU 61 has received the image data with thepriority tag in the first window 13 (step S32). As illustrated in FIG.9, this message 91 has a button 92 to stop updating the image datadisplayed in the first window 13, and a button 93 to direct the CPU 61to update the image data displayed in the first window 13.

Then, the CPU 61 determines whether the direction of updating the imagedata displayed in the first window 13 is being input (step S33). Whenthe determination of the step S33 is YES, the CPU 61 displays the imagedata with the priority tag in the first window 13 (step S34), and goesback to the step S31. Meanwhile, when the determination of the step S33is NO, the CPU 61 keeps displaying the image data displayed in the firstwindow 13 (step S35), and goes back to the step S31.

FIG. 10 is a flowchart showing a process executed with the video server2.

First, the CPU 21 of the video server 2 determines whether it isreceiving the termination request (e.g. a request to shut down a power)(step S41). When the determination of the step S41 is YES, the presentprocess is terminated. Meanwhile, when the determination of the step S41is NO, the CPU 21 determines whether it is acquiring the new image datafrom the first camera 1 or the second camera 9 (step S42).

When the determination of the step S42 is NO, the CPU 21 goes back tothe step S41. Meanwhile, when the determination of the step S42 is YES,the CPU 21 determines whether it is receiving the request which requeststo add the priority tag to the acquired image data (step S43). Therequest which requests to add the priority tag to the image data isspecified with the mouse and keyboard 25 that are connected to the videoserver 2. Or it is specified by the access to the video server 2 fromthe PC 6. Also it should be noted that the request which requests to addthe priority tag to the image data may be specified automatically atpredetermined intervals (e.g. ten seconds) based on the order of theprogram stored in HDD 24.

When the determination of the step S43 is YES, the CPU 21 adds thepriority tag to the acquired image data (step S44), and temporarilystores the image data with the priority tag in the HDD 24 (step S45).Meanwhile, when the determination of the step S43 is NO, the CPU 21moves to the step S45. In this case, the image data temporarily storedin HDD 24 is not being added the priority tag to.

Then, the CPU 21 determines whether it is receiving the request of theimage data temporarily stored in the HDD 24 from the distribution server3 (step S46). When the determination of the step S46 is YES, the CPU 21supplies the data image temporarily stored in the HDD 24 to thedistribution server 3 (step S47), and goes back to the step S41.Meanwhile when the determination of the step S46 is NO, the CPU 21 goesback to the step S41.

FIG. 11 is a flowchart showing the process executed with thedistribution server 3.

First, the CPU 31 of the distribution server 3 determines whether it isreceiving the termination request (e.g. a request to shut down a power)(step S51). When the determination of the step S51 is YES, the presentprocess is terminated. Meanwhile when the determination of the step S51is NO, the CPU 31 determines whether it is receiving the request of theimage data from the PC 6 (step S52).

When the determination of the step S52 NO, the CPU 31 goes back to thestep S51. Meanwhile, when the determination of the step S52 is YES, theCPU 31 requests the image data temporarily stored in the HDD 24 to thevideo server 2 (step S53), supplies the image data received from thevideo server 2 to the PC 6 (step S54), and goes back to the step S51.

FIG. 12 is a flowchart showing the process executed with the drawingserver 4.

The CPU 41 of the drawing server 4 waits until it receives a commandfrom the PC 6 (step S61). When the CPU 41 receives the commandpresenting the annotation image from the PC 6 (step S62), the CPU 41converts the command to the data for the projector 8, and outputs it(step S63). The projector 8 projects the annotation image onto theobject 11 based on the converted data.

Then, the CPU 41 determines whether it is receiving the terminationrequest (e.g. a request to shut down a power) (step S64). When thedetermination of the step S64 is YES, the present process is terminated.Meanwhile, when the determination of the step S64 is NO, the CPU 41 goesback to the step S61.

As described in detail above, according to the exemplary embodiment, theCPU 61 starts to receive the several image data in parallel from theserver system 1 to which the first camera 7 and the second camera 9 thatcapture the object are connected, detects the image data with thepriority tag to be preferentially displayed on the display device 12from the several image data being received, stops receiving the imagedata from which the priority tag hasn't been detected, and displays theimage data from which the priority tag has been detected on the displaydevice 12. Therefore, the specified image data is preferentiallydisplayed regardless of the reception condition of the image data. Alsoa communication band of the network is used efficiently because thereception of the image data from which the priority tag is not detectedis stopped.

It should be noted that the CPU 61 may display the buttons 92 and 93 tochoose whether a user stops updating the image data displayed on thedisplay device 12 when the image data with the priority tag has beenreceived. In addition it should be noted that before that the CPU 61 maydisplay information showing that the CPU 61 has received the image datawith the priority tag. In these cases, the user can figure out thedisplay of the priority image, and choose whether or not to stopupdating the captured image displayed on the display device 12.

A recording medium on which the software program for realizing thefunctions of the PC 6 is recorded may be supplied to the PC 6, and theCPU 61 of the PC 6 may read and execute the program recorded on therecording medium. In this manner, the same effects as those of theabove-described exemplary embodiment can be achieved. The recordingmedium for providing the program may be a CD-ROM, a DVD, or a SD card,for example.

Alternatively, the CPU 61 of the PC 6 may execute a software program forrealizing the functions of each PC, so as to achieve the same effects asthose of the above-described exemplary embodiment.

It should be noted that the present invention is not limited to thoseexemplary embodiments, and various modifications may be made to themwithout departing from the scope of the invention.

1. An information processing apparatus comprising: a reception unit that receives multiple captured images that are captured with a capture device in parallel from an external apparatus to which the capture device that captures an object is connected; a detection unit that detects a priority image that is included in the multiple captured images being received by the reception unit and is assigned priority information that specifies the priority image to be preferentially displayed on a display device; and a controller that controls the reception unit to stop receiving other captured images which are not detected by the detection unit and to receive the priority image detected by the detection unit and controls the display device to display the priority image on the display device when the priority image is detected by the detection unit.
 2. The information processing apparatus according to claim 1, wherein the detection unit detects the priority image with the priority information from header information of each of the multiple captured images received by the reception unit.
 3. The information processing apparatus according to claim 1, wherein the controller controls the display device to display selection information to choose whether or not a user stops updating an image displayed on the display device to the priority image when the reception unit receives the priority image.
 4. A remote indication system comprising: a first information processing apparatus including a first reception unit that receives multiple captured image from a capture device that captures an object, and an addition unit that adds priority information that specifies a priority image displayed preferentially on a display device to one or more of the multiple captured image received by the first reception unit; and a second information processing apparatus including a second reception unit that is connected to the first information processing apparatus via a communication medium, and receives the multiple captured image in parallel from the first information processing apparatus, a detection unit that detects the priority image from the multiple captured images received by the second reception unit, and a controller that controls the second reception unit to stop receiving other captured images which are not detected by the detection unit and to receive the priority image detected by the detection unit and controls the display device to display the priority image on the display device when the priority image is detected by the detection unit.
 5. The remote indication system according to claim 4, wherein the addition unit adds the priority information to a captured image selected by the first information processing apparatus or the second information processing apparatus.
 6. A computer readable medium causing a computer to execute a process, the process comprising: receiving multiple captured images by a capture device in parallel from an external apparatus to which the capture device that captures an object is connected; detecting a priority image that is included in the multiple captured images being received and is assigned priority information that specifies the priority image to be preferentially displayed on a display device; and stopping receiving other captured images not assigned the priority information, and receiving the priority image with the priority information to display the priority image on the display device when the priority image is detected. 